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Related Applications 

[0001] This application hereby claims priority under 35 U.S.C. §1 19 to 
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entitled, "Overdetermined (Tall) Systems of Nonlinear Equations," by inventors 
G. William Walster and Eldon R. Hansen (Attorney Docket No. SUN-8507PSP). 

BACKGROUND 

Field of the Invention 

[0002] The present invention relates to techniques for performing 
arithmetic operations involving interval operands within a computer system. 
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More specifically, the present invention relates to a method and an apparatus for 
computing interval parameter bounds from fallible measurements using systems 
of nonlinear equations. 

Related Art 

[0003] Rapid advances in computing technology make it possible to 
perform trillions of computational operations each second. This tremendous 
computational speed makes it practical to perform computationally intensive tasks 
as diverse as predicting the weather and optimizing the design of an aircraft 
engine. Such computational tasks are typically performed using machine- 
representable floating-point numbers to approximate values of real numbers. (For 
example, see the Institute of Electrical and Electronics Engineers (IEEE) standard 
754 for binary floating-point numbers.) 

[0004] In spite of their limitations, floating-point numbers are generally 
used to perform most computational tasks. 

[0005] One limitation is that machine-representable floating-point 
numbers have a fixed-size word length, which limits their accuracy. Note that a 
floating-point number is typically encoded using a 32, 64 or 128-bit binary 
number, which means that there are only 2 32 , 2 64 or 2 128 possible symbols that can 
be used to specify a floating-point number. Hence, most real number values can 
only be approximated with a corresponding floating-point number. This creates 
estimation errors that can be magnified through even a few computations, thereby 
adversely affecting the accuracy of a computation. 

[0006] A related limitation is that floating-point numbers contain no 
information about their accuracy. Most measured data values include some 
amount of error that arises from the measurement process itself. This error can 
often be quantified as an accuracy parameter, which can subsequently be used to 
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determine the accuracy of a computation. However, floating-point numbers are 
not designed to keep track of accuracy information, whether from input data 
measurement errors or machine rounding errors. Hence, it is not possible to 
determine the accuracy of a computation by merely examining the floating-point 
number that results from the computation. 

[0007] Interval arithmetic has been developed to solve the above- 
described problems. Interval arithmetic represents numbers as intervals specified 
by a first (left) endpoint and a second (right) endpoint. For example, the interval 
[a, b] 9 where a < 6, is a closed, bounded subset of the real numbers, /?, which 
includes a and b as well as all real numbers between a and b. Arithmetic 
operations on interval operands (interval arithmetic) are defined so that interval 
results always contain the entire set of possible values. The result is a 
mathematical system for rigorously bounding numerical errors from all sources, 
including measurement data errors, machine rounding errors and their 
interactions. (Note that the first endpoint normally contains the "infimum", which 
is the largest number that is less than or equal to each of a given set of real 
numbers. Similarly, the second endpoint normally contains the "supremum", 
which is the smallest number that is greater than or equal to each of the given set 
of real numbers. Also note that the infimum and the supremum can be 
represented by floating point numbers.) 

[0008] One commonly performed operation is to compute bounds on 
nonlinear parameters from a set of fallible measurements. Using the traditionally 
accepted methodology to compute approximate parameter values from nonlinear 
models of observable data requires a number of questionable assumptions. In the 
best case, if all assumptions are satisfied, the final result is a less than 100% 
statistical confidence interval rather than a containing interval bound. For 
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example, the method of least squares produces a solution approximation even 
when the data on which it is based are inconsistent. 

[0009] Hence, what is needed is a method and an apparatus that uses 
interval techniques to compute bounds on nonlinear parameters from fallible 
measurements. 

SUMMARY 

[0010] One embodiment of the present invention provides a system that 
computes interval parameter bounds from fallible measurements. During 
operation, the system receives a set of measurements z u ... 9 z„ 9 wherein an 
observation model describes each z, as a function of a ^-element vector parameter 
x = (x\ 9 . . . , x p ). Next, the system forms a system of nonlinear equations 
zt - h{\) = 0 (7=1 , . . n) based on the observation model. Finally, the system 
solves the system of nonlinear equations to determine interval parameter bounds 
on x. 

[0011] In a variation on this embodiment, the system of nonlinear 
equations is an "overdetermined system" in which there are more equations than 
unknowns. 

[0012] In a variation on this embodiment, each measurement z, is actually 
a ^-element vector of measurements z, = (z iU . .. 5 z,/, and h is actually a 
^-element vector of functions h = (h u . . h q ) T . 

[0013] In a variation on this embodiment, receiving the set of 
measurements involves receiving values for a set of conditions c u . . c n under 
which the corresponding observations z, were made. In this variation, the system 
of nonlinear equations is of the form z t -h{\ \ c,) = 0 (/=1, n). 

[0014] In a further variation, each condition c, is actually an r-element 
vector of conditions Cj = (c,i, . . ., c ir ) T . 
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[0015] In a further variation, each condition c, is not known precisely but 
is contained within an interval c 1 ,. 

[0016] In a variation on this embodiment, equations in the system of 
nonlinear equations are of the form z\ - h(\ \ c,) + £ ! (x, c,) = 0 (/=1 , . . . , n), which 
includes an error model e\x, c t ) that provides interval bounds on measurement 
errors for z h 

[0017] In a further variation, if z, is actually a ^-element vector of 
measurements z ; = (z n , . . ., z iq ) T , then s x is actually a ^-element vector 
e'^O,, ...,€ q ) T . 

[0018] In a further variation, if there exists no solution to the system of 
nonlinear equations, the system determines that at least one of the following is 
true: (1) at least one of the set of measurements z„ . . ., z n is faulty; (2) the 
observation model h(x | c,) is false; (3) the error model € ! (x, c 7 ) is false; and (4) 
the computational system used to compute interval bounds on elements of x is 
flawed. 

[0019] In a variation on this embodiment, solving the system of nonlinear 
equations involves first linearizing the system of nonlinear equations to form a 
corresponding system of linear equations, and then solving the system of linear 
equations through Gaussian elimination. 

BRIEF DESCRIPTION OF THE FIGURES 
[0020] FIG. 1 illustrates a computer system in accordance with an 
embodiment of the present invention. 

[0021] FIG. 2 illustrates the process of compiling and using code for 
interval computations in accordance with an embodiment of the present invention. 

[0022] FIG. 3 illustrates an arithmetic unit for interval computations in 
accordance with an embodiment of the present invention. 

5 

Attorney Docket No. SUN-P8507-SPL Inventors: Walster et at. 

ARP H:\SUN MICROSYSTEMS\SUN-P8507-SPL\SUN-P8507-SPL APPLICATION. DOC 



[0023] FIG. 4 is a flow chart illustrating the process of performing an 
interval computation in accordance with an embodiment of the present invention. 

[0024] FIG. 5 illustrates four different interval operations in accordance 
with an embodiment of the present invention. 

[0025] FIG. 6 illustrates the process of performing a Gaussian Elimination 
operation on an overdetermined interval system of linear equations in accordance 
with an embodiment of the present invention. 

[0026] FIG. 7 illustrates the process of generating a preconditioning 
matrix in accordance with an embodiment of the present invention. 

[0027] FIG. 8 presents a flow chart illustrating the process of computing 
interval parameter bounds from fallible measurements in accordance with an 
embodiment of the present invention. 

[0028] Table 1 (located near the near the end of the specification-not with 
the figures) illustrates a correspondence between parameter estimation and 
nonlinear equations in accordance with an embodiment of the present invention. 

DETAILED DESCRIPTION 
[0029] The following description is presented to enable any person skilled 
in the art to make and use the invention, and is provided in the context of a 
particular application and its requirements. Various modifications to the disclosed 
embodiments will be readily apparent to those skilled in the art, and the general 
principles defined herein may be applied to other embodiments and applications 
without departing from the spirit and scope of the present invention. Thus, the 
present invention is not intended to be limited to the embodiments shown, but is 
to be accorded the widest scope consistent with the principles and features 
disclosed herein. 
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[0030] The data structures and code described in this detailed description 
are typically stored on a computer readable storage medium, which may be any 
device or medium that can store code and/or data for use by a computer system. 
This includes, but is not limited to, magnetic and optical storage devices such as 
disk drives, magnetic tape, CDs (compact discs) and DVDs (digital versatile discs 
or digital video discs), and computer instruction signals embodied in a 
transmission medium (with or without a carrier wave upon which the signals are 
modulated). For example, the transmission medium may include a 
communications network, such as the Internet. 

Computer System 

[0031] FIG. 1 illustrates a computer system 100 in accordance with an 
embodiment of the present invention. As illustrated in FIG. 1, computer system 
100 includes processor 102, which is coupled to a memory 1 12 and a to peripheral 
bus 1 10 through bridge 106. Bridge 106 can generally include any type of 
circuitry for coupling components of computer system 100 together. 

[0032] Processor 102 can include any type of processor, including, but not 
limited to, a microprocessor, a mainframe computer, a digital signal processor, a 
personal organizer, a device controller and a computational engine within an 
appliance. Processor 102 includes an arithmetic unit 104, which is capable of 
performing computational operations using floating-point numbers. 

[0033] Processor 102 communicates with storage device 108 through 
bridge 106 and peripheral bus 110. Storage device 108 can include any type of 
non-volatile storage device that can be coupled to a computer system. This 
includes, but is not limited to, magnetic, optical, and magneto-optical storage 
devices, as well as storage devices based on flash memory and/or battery-backed 
up memory. 
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[0034] Processor 102 communicates with memory 1 12 through bridge 
106. Memory 1 12 can include any type of memory that can store code and data 
for execution by processor 102. As illustrated in FIG. 1, memory 1 12 contains 
computational code for intervals 1 14. Computational code 1 14 contains 
instructions for the interval operations to be performed on individual operands, or 
interval values 115, which are also stored within memory 1 12. This 
computational code 1 14 and these interval values 1 15 are described in more detail 
below with reference to FIGs. 2-5. 

[0035] Note that although the present invention is described in the context 
of computer system 100 illustrated in FIG. 1, the present invention can generally 
operate on any type of computing device that can perform computations involving 
floating-point numbers. Hence, the present invention is not limited to the 
computer system 100 illustrated in FIG. 1. 

Compiling and Using Interval Code 

[0036] FIG. 2 illustrates the process of compiling and using code for 
interval computations in accordance with an embodiment of the present invention. 
The system starts with source code 202, which specifies a number of 
computational operations involving intervals. Source code 202 passes through 
compiler 204, which converts source code 202 into executable code form 206 for 
interval computations. Processor 102 retrieves executable code 206 and uses it to 
control the operation of arithmetic unit 104. 

[0037] Processor 102 also retrieves interval values 115 from memory 1 12 
and passes these interval values 1 15 through arithmetic unit 104 to produce results 
212. Results 212 can also include interval values. 

[0038] Note that the term "compilation" as used in this specification is to 
be construed broadly to include pre-compilation and just-in-time compilation, as 
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well as use of an interpreter that interprets instructions at run-time. Hence, the 
term "compiler" as used in the specification and the claims refers to pre- 
compilers, just-in-time compilers and interpreters. 

Arithmetic Unit for Intervals 

[0039] FIG. 3 illustrates arithmetic unit 104 for interval computations in 
more detail accordance with an embodiment of the present invention. Details 
regarding the construction of such an arithmetic unit are well known in the art. 
For example, see U.S. Patent Nos. 5,687,106 and 6,044,454. Arithmetic unit 104 
receives intervals 302 and 3 12 as inputs and produces interval 322 as an output. 

[0040] In the embodiment illustrated in FIG. 3, interval 302 includes a 
first floating-point number 304 representing a first endpoint of interval 302, and a 
second floating-point number 306 representing a second endpoint of interval 302. 
Similarly, interval 312 includes a first floating-point number 314 representing a 
first endpoint of interval 312, and a second floating-point number 316 
representing a second endpoint of interval 312. Also, the resulting interval 322 
includes a first floating-point number 324 representing a first endpoint of interval 
322, and a second floating-point number 326 representing a second endpoint of 
interval 322. 

[0041] Note that arithmetic unit 104 includes circuitry for performing the 
interval operations that are outlined in FIG. 5. This circuitry enables the interval 
operations to be performed efficiently. 

[0042] However, note that the present invention can also be applied to 
computing devices that do not include special-purpose hardware for performing 
interval operations. In such computing devices, compiler 204 converts interval 
operations into a executable code that can be executed using standard 
computational hardware that is not specially designed for interval operations. 
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[0043] FIG. 4 is a flow chart illustrating the process of performing an 
interval computation in accordance with an embodiment of the present invention. 
The system starts by receiving a representation of an interval, such as first 
floating-point number 304 and second floating-point number 306 (step 402). 
5 Next, the system performs an arithmetic operation using the representation of the 
interval to produce a result (step 404). The possibilities for this arithmetic 
operation are described in more detail below with reference to FIG. 5. 

Interval Operations 

10 [0044] FIG. 5 illustrates four different interval operations in accordance 

with an embodiment of the present invention. These interval operations operate 
on the intervals X and Y. The interval X includes two endpoints, 

x denotes the lower bound of X, and 
x denotes the upper bound of X. 

[0045] The interval X is a closed subset of the extended (including -«> and 
15 +<») system of real numbers i?* (see line 1 of FIG. 5). Similarly the interval Falso 
has two endpoints and is a closed subset of the extended real numbers R* (see 
line 2 of FIG. 5). 

[0046] Note that an interval is a point or degenerate interval if X= [x, x]. 
Also note that the left endpoint of an interior interval is always less than or equal 
20 to the right endpoint. The set of extended real numbers, /?* is the set of real 
numbers, R, extended with the two ideal points negative infinity and positive 
infinity: 

R* = (RU {-oo}) U {+~} = [-oo,+oo]. 

25 
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[0047] We also define R** by replacing the unsigned zero, {0} 5 from R* 
with the interval [-0,+0]. 

i?** = i?* - {0} u [-0,+0] = [-~ 5 +°°] 5 because 0 = [-0,+0]. 

5 

[0048] In the equations that appear in FIG. 5 5 the up arrows and down 
arrows indicate the direction of rounding in the next and subsequent operations. 
Directed rounding (up or down) is applied if the result of a floating-point 
operation is not machine-representable. 
10 [0049] The addition operation X+ Y adds the left endpoint of Xto the left 

endpoint of Y and rounds down to the nearest floating-point number to produce a 
resulting left endpoint, and adds the right endpoint of ^to the right endpoint of Y 
and rounds up to the nearest floating-point number to produce a resulting right 
endpoint. 

1 5 [0050] Similarly, the subtraction operation X-Y subtracts the right 

endpoint of Y from the left endpoint of X and rounds down to produce a resulting 
left endpoint, and subtracts the left endpoint of Ffrom the right endpoint of X and 
rounds up to produce a resulting right endpoint. 

[0051] The multiplication operation selects the minimum value of four 

20 different terms (rounded down) to produce the resulting left endpoint. These 
terms are: the left endpoint of X multiplied by the left endpoint of Y\ the left 
endpoint of X multiplied by the right endpoint of Y\ the right endpoint of X 
multiplied by the left endpoint of 7; and the right endpoint of ^multiplied by the 
right endpoint of Y. This multiplication operation additionally selects the 

25 maximum of the same four terms (rounded up) to produce the resulting right 
endpoint. 
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[0052] Similarly, the division operation selects the minimum of four 
different terms (rounded down) to produce the resulting left endpoint. These 
terms are: the left endpoint of X divided by the left endpoint of Y\ the left endpoint 
of X divided by the right endpoint of Y; the right endpoint of X divided by the left 
5 endpoint of Y; and the right endpoint of X divided by the right endpoint of 7. This 
division operation additionally selects the maximum of the same four terms 
(rounded up) to produce the resulting right endpoint. For the special case where 
the interval Y includes zero, XI Y is an exterior interval that is nevertheless 
contained in the interval R*. 
10 [0053] Note that the result of any of these interval operations is the empty 

interval if either of the intervals, Xor Y, are the empty interval. Also note, that in 
one embodiment of the present invention, extended interval operations never 
cause undefined outcomes, which are referred to as "exceptions" in the IEEE 754 
standard. 

15 

Solving an Overdetermined System of Interval Linear Equations 

[0054] In order to solve a system of interval nonlinear equations, we first 
describe a technique for solving a system of interval linear equations. We can 
subsequently use this technique in solving a corresponding system of interval 
20 nonlinear equations. 

[0055] Given the real (n x n) matrix A and the (n x 7) column vector b, 
the linear system of equations 



25 



Ax=b (1) 

is consistent if there is a unique (n x 1) vector x for which the system in (1) is 
satisfied. If the number of rows in A and elements in b is m*n, then the system is 
said to be either under- or overdetermined depending on whether m<n or n<m. In 
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the overdetermined case, if m~n equations are not linearly dependent on the 
remaining equations, there is no solution vector x that satisfies the system. In the 
underdetermined case there is no unique solution. 

[0056] In the point (non-interval) case, there is no generally reliable way 
5 to decide if an overdetermined system based on fallible observations is consistent 
or not. Instead a least squares solution is generally sought. In the interval case, if 
the system of equations is sufficiently inconsistent, the computed interval solution 
set will be empty. If there are at least some parameter values that are consistent 
with all the observations, it is possible to delete inconsistent parameter values and 
1 0 bound the consistent ones. 

[0057] We now consider the problem of solving overdetermined systems 
of equations in which the coefficients are intervals. That is, we consider a system 
of the form 

15 AW (2) 

where A 1 is an interval matrix of m rows and n columns with m>n. The interval 
vector b 1 has m components. Such a system might arise directly or by linearizing 
an overdetermined system of nonlinear equations. (Note that within this 

20 specification and in the following claims, we sometimes drop the superscript "I" 
when referring the interval matrices or vectors.) 

[0058] The solution set of (2) is the set of vectors x for which there exists 
a real matrix A e A 1 and a real vector beb 1 such that (1) is satisfied. In general, 
the system in (2) is inconsistent if its solution set is empty. First, we assume that 

25 there exists at least one A e A 1 and beb 1 such that (1) is consistent. Later, we 
consider the inconsistent case. Moreover, we also assume that the data in A 1 and 
b l are fallible. That is, there exists at least one AeA 1 and beb 1 such that (1) is 
inconsistent. Our goal is to implicitly exclude values of x that are inconsistent 
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with all A € A 1 and b e b ! . For example, the redundancy resulting from the fact 
that there are more equations than variables might be deliberately introduced to 
sharpen the interval bound on the set of solutions to (2). In a following section, 
we show how this sharpening is accomplished. 

[0059] We shall simplify the system using Gaussian elimination. In the 
point case, it is good practice to avoid forming normal equations from the original 
system. Instead, one performs elimination using normal operation matrices to 
zero all elements of the coefficient matrix except for an upper triangle. After this 
first phase, the normal equations of this simpler system can be formed and solved. 
Our procedure begins with a phase similar to the first phase just described. 
However, we do not quite complete the usual elimination procedure. We have no 
motivation to use normal operations because we do not form the normal 
equations. This is just as well because interval normal matrices do not exist. 

[0060] When using interval Gaussian elimination, it is generally necessary 
to precondition the system to avoid excessive widening of intervals due to 
dependence. In the following section, we show how preconditioning can be done 
in the present case where A 1 is not square. 



Preconditioning 

[0061] Preconditioning can be done in the same way it is done when A 1 is 
square. Let A c denote the center of the interval matrix A 1 . Partition A c as 



(3) 
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where A c is an n by n matrix and A " c is an m-n by n matrix. Note that A c need 

only be an approximation for the center of A 1 . Define the partitioned square 
matrix 



C = 



A c 0 
A" I 



(4) 



where I denotes the identity matrix of order m-n, and the block denoted by 0 is an 
n x m-n matrix of zeros. 

[0062] Define the preconditioning matrix B to be an approximation for the 



10 inverse 



ofC. 

A C (A C )-' I 

[0063] To precondition (2) we multiply by B. We obtain 



M*x = r 1 



(5) 



15 where M 1 = BA 1 is an m by n interval matrix and r 1 = Bb 1 is an interval vector of m 
components. When computing M 1 and r\ we use interval arithmetic to bound 
rounding errors. 



20 



Elimination 

[0064] We now perform elimination. We apply an interval version of 
Gaussian elimination to the system M ! x = r l thereby transforming M l into almost 
(see below) upper trapezoidal form. We assume that this procedure only fails 
when all possible pivot elements contain zero. Note that after preconditioning, no 
pivot selection is performed during the elimination to obtain a result with the form 



25 
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T 1 




~u'~ 




x = 




w 





(6) 



where T 1 is a square upper triangular interval matrix of order /?, and both u l and v l 
are interval vectors of n and m-n components, respectively. The submatrix W 1 is a 
5 matrix of m-n rows and n columns. It is zero except in the last column. 
Therefore, we can represent it in the form 

W'=[0 z 1 ] 

10 where 0 denotes an m-n by n-\ block of zeros, and z 1 is a vector of m-n intervals. 
From (6), we now have a set of equations 

ZjX„ = Vj(i=l,...,m-n). (7) 

15 Also, 

T m x n = u n . (8) 
Therefore, the unknown value x n is contained in the interval 

Taking this intersection is what implicitly eliminates fallible data from A 1 and b 1 . 
It is this operation that allows us to get a sharper bound on the set of solutions to 
the original system (2) than might otherwise be obtained. 
25 [0065] If the original system contains at least one consistent set of 

equations, the intersection in (9) will not be empty. Knowing x n we can backsolve 
(6) for x n -i, . • . , *i . From (6), this takes the standard form of backsolving a 
triangular system T l x = u l . Sharpening x n using (9) also produces sharper bounds 
x 1 on the other components of x when we backsolve. 
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Inconsistency 

[0066] Now suppose the initial equations (2) are not consistent. Then the 
preconditions of equations (7) might or might not be consistent. Widening of 
intervals due to dependence and roundoff can cause the intersection in (9) to be 
non-empty. 

[0067] Nevertheless, suppose we find that the intersection in (9) is empty. 
This event proves that the original equations (2) are inconsistent. Proving 
inconsistency might be the signal that a theory is measurably false, which might 
be an extremely enlightening event. On the other hand, inconsistency might only 
mean that invalid measurements have been made. 

[0068] If invalid measurements are suspected, it might be important to 
discover which equation(s) in (2) are inconsistent. We might know which 
equation(s) in the transformed system (6) must be eliminated to obtain 
consistency. However, an equation in (6) is generally a linear combination of all 
the original equations in (2). Therefore, to establish consistency in the original 
system, we generally cannot determine which of its equation(s) to remove. 

[0069] We might be able to determine a likely removal candidate by using 
the following steps: 

1. Remove enough equations from (6) that the intersection in (9) is not 
empty. 

2. Solve (6) for x n _i, . . . 5 x\. This process cannot fail because we assume the 
elimination process to obtain (6) does not fail. 

3. Substitute the solution into the original system (2). Any equation(s) in (2) 
whose left and right members do not intersect can be discarded. 
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Summary of the Gaussian Elimination Operation 

[0070] FIG. 6 illustrates the process of performing a Gaussian Elimination 
operation on an overdetermined interval system of linear equations in accordance 
with an embodiment of the present invention. The system starts by receiving a 
representation of the overdetermined system of linear equations Ax=b (step 602). 
In this representation, A is a matrix with m rows corresponding to m equations 
and n columns corresponding to n variables, x includes n variable components, b 
includes m scalar components, and m > n. The system then stores this 
representation in memory (step 604). 

[0071] Next, the system preconditions Ax = b to generate a modified 
system BAx = Bb that can be solved with reduced growth of interval widths 
(step 606). This preconditioning process is described in more detail below with 
reference to FIG. 7. 

[0072] The system then performs an interval Gaussian elimination 

, wherein T is a square upper 

triangular matrix of order n, u is an interval vector with n components, v is an 
interval vector with m-n components, and W is a matrix with m-n rows and n 
columns, and wherein W is zero except in the last column, which is represented as 
a column vector z with m-n components (step 608). 

[0073] Note that interval Gaussian elimination can fail. If so, the system 
simply terminates (step 609). 

[0074] If Gaussian elimination does not fail, the system performs an 
interval intersection operation based on the equations z, x„ = v, (/=1 ,. . .,m-ri) and 

T m *n = u n to solve for x n = Q-^ (step 610). 



operation on BAx=Bb to form 



T 

x - 

W 
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[0075] Finally, if x n is not the empty interval, the system performs a back 
substitution operation using x n and Tx=u to solve for the remaining components 
(x n _i 5 ... 5 xj)ofx (step 612). 

[0076] FIG. 7 illustrates the process of generating a preconditioning 
matrix in accordance with an embodiment of the present invention. The system 
starts by determining a non-interval matrix A C5 which is the approximate center of 
the interval matrix A (step 702). Next, the system augments the m x n matrix A c 



to produce an n x n partitioned matrix C = 



A c 0 



wherein A' is an n x n 



matrix, A c is an m-n x n matrix, I is the identity matrix of order m-n, and 0 is an 
n x m-n matrix of zeros (step 704). Finally, the system calculates the approximate 
inverse of the partitioned matrix C to produce the preconditioning matrix B 
(step 706). If C happens to be singular, its elements can be perturbed until it is no 
longer so. This causes no difficulty because C is just used to compute the 
approximate inverse B. 

Parameter Estimation in Nonlinear Models 

[0077] Overdetermined (tall) systems of nonlinear equations naturally 
arise in the context of computing interval parameter bounds from fallible data. In 
tall systems, there are more interval equations than unknowns. As a result, these 
systems can appear to be inconsistent when they are not. A technique is described 
to compute interval nonlinear parameter bounds from fallible data and to possibly 
prove that no bounds exist because the tall system is inconsistent. 

[0078] Interval arithmetic has been used to perform the analysis of fallible 
observations from an experiment to compute bounds on Newton's constant of 
gravitation G. (see B. Lang, Verified Quadrature in Determining Newton's 
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Constant of Gravitation, Journal of Universal Computer Science, 4(1): 16-24, 
1998.) Because the computed bounds were sufficiently different from the then 
accepted approximate value, subsequent experiments were conducted to refine the 
accepted approximate value and the interval bound on G. (see "The Controversy 
of Newton's Gravitational Constant," The Eot-Wash Group: Laboratory of 
Gravitational Physics, www.npl.washington.edu/eotwash/gconst.html) 

[0079] Using the traditionally accepted methodology to compute 
approximate parameter values from nonlinear models of observable data requires 
a number of questionable assumptions. In the best case, if all assumptions are 
satisfied, the final result is a less than 100% statistical confidence interval rather 
than a containing interval bound. For example, the method of least squares 
produces a solution approximation even when the data on which it is based are 
inconsistent. 

[0080] A better procedure is to solve a system of interval nonlinear 
equations using the interval version of Newton's method. If assumptions for this 
procedure are satisfied, the result is a guaranteed bound on the parameter(s) in 
question. If assumptions are sufficiently violated and enough observations are 
available, the procedure can prove the system of equations and interval data are 
inconsistent. This better procedure is now described. 

Nonlinear Parameter Estimation 

[0081] Let n ^-element vector measurements, Zi, . . ., z n with 
Zj - (z/i, . . z iq f be given. Assume these measurements depend on the value of a 
/^-element vector parameter, x, = (x h . ..,x p ) T . Moreover, assume an analytic 
model exists for the observation vectors, z„ as a function of x and the true value 
c, = (en, c ir ) T of conditions under which the z,- are measured. Thus: 

z/ = h(x|c / ) (11) 
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The problem is to construct interval bounds x 1 on the elements of x from interval 
bounds z 1 , on the fallible measurements, z, and interval bounds c\ on the 
conditions of measurement. 

5 

Interval Observation Bounds 

[0082] The development of interval measurement bounds begins by 
recognizing that a measurement z can be modeled (or thought of) as an unknown 
value t to which an error is added from the interval 

10 

ex [-1,1] = e l (12) 

where 0 < e. No assumption is made about the distribution of individual 
measurement errors from the interval € [ that are added to t in the process of 
1 5 measuring z. 

[0083] At once it follows that 

zet+e\ (13) 
20 More importantly, if the interval observation Z is defined to be z + e\ then 

teZ. (14) 

Enclosure (14) is an immediate consequence of the fact that zero is the midpoint 
25 off 1 . This simple idea has a number of implications. They are: 

• Given multiple interval observations Z„ all of which are enclosures off, so 

n 

must their intersection. Therefore t e C\Z i f . 
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• Given random finite intervals Z„ all of which contain the value t, the 
expected width of their intersection decreases as n increases. 

* An empty intersection is proof that t <£ Z, for some value of /. This can be 
true either because 

- the width of the interval observations Z, is too narrow, 

- there is no single value / that is contained in all the interval 
measurements Z„ or 

- both of the above. 

The first alternative means that the assumption regarding the accuracy of the 
measurement process is false. The second alternative means that the model 
for the single common value / is false. 

[0084] Walster explored how this simple idea works in practice to 
compute an interval bound on a common value under various probability 
distributions for values of the random variable s e s l . (see G. W. Walster, 
"Philosophy and Practicalities of Interval Arithmetic," R. E. Moore Editor, 
Reliability on Computing, pages 309-323, Academic Press, Inc., San Diego, 
California, 1988.) He also discussed how this estimation principle can be 
generalized and used to bound parameters of nonlinear models given bounded 
interval observations, or observation vectors. The following is a more complete 
elaboration of the nonlinear generalization. 

General Development 

[0085] Given exact values of a set of conditions c, = (c,i, . . ., c ir ) T under 
which observations z,- = (z,i, z /<7 ) r are made, assume the observation vectors, 
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z, (i=l 5 /?), satisfy the following model: 

iseh(x|c,) + e(x s cOx[-l f l]; (15) 

5 where the vectors 0 < e(x, c) bound unknown modeling and direct measurement 
errors. Specifically, if the p elements of x and the rn elements of all the c, were 
known (in practice they are not), assume it would be possible to compute intervals 

^(x, c/) = e(x, c,) x [-1,1]; (16) 

10 

from which it follows immediately that 

0 g z, - h(x | c,) + e'(x, c,). (17) 

15 [0086] Note that (16) is a generalization of (12), and (17) is a 

generalization of (14) if written in the form 0 e Z-t. 

[0087] If (as is normally the case) the conditions c, under which 
measurements z/ are made are not known, but are contained in intervals c 1 ,, then 
taking all bounded modeling and observation errors into account: 

20 

Ogz, - h(x | c 1 ,) + s [ (x, c 1 ,). (18) 

[0088] In this general form, the widths of interval measurements z 1 , are 
themselves functions of both the unknown parameters x and fallibly measured 
25 conditions under which the measurements are made. That is: 

z\ = zz + eW,). (19) 

This interval observation model is the generalization of Z = z + e x which is 
30 consistent with (13). The interval observation model (19) is needed to solve for 
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10 



interval bounds on the parameter vector x. If there is no solution for a given set of 
interval observation vectors z\ and interval bounds on measurement conditions c 1 ,, 
then either: 

• the observation model h(x | c 1 ,) is false; 

• the measurement error model e^x, c 1 ,) is false; 

• the computational system used to compute interval bounds on the elements 
of x is flawed; or 

• some combination of the above. 

In this way, and by eliminating alternative explanations, the theory represented in 
h(x | c 1 ;) or the observation error model represented in £ ! (x, c 1 ,) can be proved to 
be false. 



15 The System of Nonlinear Equations 

[0089] To guarantee any computed interval x 1 is indeed a valid bound on 
the true value of x, the following must be true: 

• the given model h; 

20 ■ the interval bounds c 1 , on the conditions under which fallible measurements z, 

are made; and, 

• the model for interval bounds e ! (x, c\) on observation errors. 

[0090] To be consistent with the given models, all the actual measurement 
25 vectors z, must satisfy relation (18). A logically equivalent, but more suggestive 
way to write this system of constraints is: 

z, - h(x | c 1 ,) + e'(x, c 1 ,) = 0 (/=1, ...,/!) (20) 
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[0091] When used in (20), a possible value of x produces intervals that 
contain zero for all /. Any value of x that fails to do this cannot be in the solution 
set of (20). Thus, (20) is just an interval system of nonlinear equations in the 
5 unknown parameter vector x. The problem is that the total number of scalar 
equations nq might be much larger than the number p of scalar unknowns in the 
parameter vector x. Point (rather than interval) systems of equations where p<nq 
are called "overdetermined". For interval nonlinear equations, this is a misnomer 
because the interval equations might or might not be consistent. As mentioned 
1 0 above, inconsistency (an empty solution set) is an informative event. 

Solving Nonlinear Equations 

[0092] Let f : R n -> R m (n < m) be a continuously differentiable function. 
The parameter estimation problem described above is just a special case of the 
15 more general problem now considered. Table 1 below shows the correspondence 
between the parameter estimation problem and equivalent nonlinear equations to 
be solved. Both unknowns and equations are shown. 





Parameter Estimation 


Nonlinear Equations 


Unknowns 


x 1 = (x\, ...,x l p) T 


x 1 = (X\, ... 9 X P ) T 


Equations 


Zi-h(x\c\) + sXx, c I l ) = 0(/=l 5 


f = 


(fu ...,f m ) T =Q where m = nq 


Table 1: Correspondence between Parameter Estimat 


ion a 


nd Nonlinear Equations 



20 

Having established this correspondence, the problem becomes to find and bound 
all the solution vectors of f(x) = 0 in a given initial box x I(0) . For non-interval 
methods, it can sometimes be difficult to find reasonable bounds on a single 
solution, quite difficult to find reasonable bounds on all solutions, and generally 
25 impossible to know whether reasonable bounds on all solutions have been found. 
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In contrast, it is a straightforward problem to find reasonable bounds on solutions 
in x I(0) using interval methods; and it is trivially easy to computationally determine 
that all solutions in x I(0) have been bounded. What is unusual in this problem is 
that the order m of f can be greater than the order p of x 1 . A factor that simplifies 
obtaining solution(s) is the assumption that the equations are consistent. This has 
the effect of reducing the number of equations at a solution to the number of 
variables. 



Linearization and Gaussian Elimination 

[0093] Let x and y be points in a box x 1 . Suppose we expand each 
component f t (z = 1 5 . . . , m) of f by one of the procedures commonly used to 
linearize nonlinear equations to be solved using the interval Newton method. 
Define the matrix of partial derivatives of the elements f t of f with respect to the 
elements xj (/=! , . ..,/?) of x: 



ax j , 



If n = m, the system is square, and J is the Jacobian of f. This is the usual 
situation in which the interval Newton method is applied, (see [Hansen] E. R. 
Hansen, "Global Optimization Using Interval Analysis," Marcel Dekker, Inc., 
New York, 1992). 

[0094] In passing it is worth noting that in place of partial derivatives, 
slopes can be used to good advantage. Slopes have narrower width than interval 
bounds on derivatives and might exist when derivatives are undefined. 
Nevertheless, the remaining development uses derivatives as they are more 
familiar than slopes. 
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[0095] Combining the results in vector form: 

f(y)€f(x) + J(x,x ! )(y-x) (21) 

Even in the non-square situation, J is still referred to herein as the Jacobian off. 
The notation J(x, x 1 ) is used to emphasize the fact that a tighter expansion of/can 
be obtained if both point and interval values of x elements are used to compute 
Jacobian matrix elements (see [Hansen]). 

[0096] If y is a zero off, then f(y) = 0, and (21) is replaced by, 

f(x) + J(x,x , )(y-x) = 0. (22) 
Define the solution set of (22) to be 

s = {y \m + [J(x, x'(/))(y-x)] / = 0, x'(0 e x 1 ...,*)}. 

This set contains any point y € x 1 for which f(y) = 0. 

[0097] The smaller the box x 1 , the smaller the set s. The object of an 
interval Newton method is to reduce x 1 until s is as small as desired so that a 
solution point y e x 1 is tightly bounded. Note that s is generally not a box. 

[0098] Normally, the system of linear equations in (22) is solved using any 
of a variety of interval methods. In the present situation if n < m, the linear 
system is overdetermined and therefore appears to be inconsistent. This is not 
necessarily the case. If the procedure described above for interval linear equations 
is used to compute an interval bound y 1 on y, then y 1 contains the set of consistent 
solutions s. 

[0099] For the solution of (22), the standard and distinctive notation 
N(x, x 1 ) is used in place of y 1 . This emphasizes the solution's dependence on both 
x and x 1 . 
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[00100] 



From (22), define an iterative process of the form 



f(x) + J(x, x'XNCxW, x m ) - x) = 0 (23a) 
X «*H) =X K*> fl N (x ( *>, x 1( < } ) (23b) 

for k = 0, 1 ,2, • • ■ where x w must be in x K *\ A good choice for x w is the center 
m(\ m ) of x I(A) . For details on computing N (\ {k \ x m ) when the system of interval 
linear equations appears to be overdetermined, see [Hansen]. 

Summary of Parameter Estimation in Nonlinear Models 
[00101] FIG. 8 presents a flow chart summarizing the process of 
computing interval parameter bounds from fallible measurements in accordance 
with an embodiment of the present invention. During operation, the system 
receives a set of measurements z u ... ,z n (step 802), as well as values for 
measurement conditions Ci, . . ., c„ under which the corresponding observations z,- 
were made (step 804). Next, the system forms a system of interval nonlinear 
equations zj - h(x | c,) + e^x, c,) = 0 (i=l , . . n) based on a nonlinear model h and 
an error model e (step 806). 

[00102] The system then uses standard interval Newtown techniques 
described above to solve the system of nonlinear equations to determine interval 
parameter bounds for x. More specifically, the system linearizes the system of 
nonlinear equations (step 808), and then solves the system of linear equations 
using the technique described above with reference to FIG. 6 (step 810). The 
system then intersects the solution with the given box (step 812). Next, the 
system determines if the solution has converged to be within specified tolerances 
(step 814). If so, the system stops. Otherwise, the system applies the interval 
Newton procedure for splitting if needed (step 816) and returns to step 808 to 
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linearize the system of equations again. 



Conclusion for Parameter Estimation in Nonlinear Models 

[00103] Computing bounds on nonlinear parameters from fallible 
observations is a pervasive problem. In the presence of uncertain observations, 
attempting to capture uncertainty with Gaussian error distributions is problematic 
when nonlinear functions of observations are computed. 

[00104] The procedure described in this specification uses the interval 
solution of a system of nonlinear equations to compute bounds on nonlinear 
parameters from fallible data. Among the many advantages of this approach is the 
ability to aggregate data from independent experiments, thereby continuously 
narrowing interval bounds. Whenever different interval results are inconsistent, 
or if the set of interval bounds from a given data set is empty, this proves an 
assumption is violated or the model for the observations is measurably wrong. 

[00105] Narrower parameter bounds can be computed from a calibrated 
system. It is interesting to note that the same procedure as described above can be 
used to solve the calibration problem. All that must be done is to modify (8) in 
the following way: 

• replace selected unknown parameter values with their now measured bounds 
x 1 ,; and 

• solve for narrower bounds on any parameters in the model for e ! (x, c,). 

[00106] The foregoing descriptions of embodiments of the present 
invention have been presented for purposes of illustration and description only. 
They are not intended to be exhaustive or to limit the present invention to the 
forms disclosed. Accordingly, many modifications and variations will be apparent 
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to practitioners skilled in the art. Additionally, the above disclosure is not 
intended to limit the present invention. The scope of the present invention is 
defined by the appended claims. 
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